Scalable resource management in distributed environment

ABSTRACT

A method may include receiving a packet in a network device, selecting one of a group of ingress buffers, where each ingress buffer is associated with a different one of a group of processors, distributing the packet to the selected ingress buffer; and scheduling the packet, based on a congestion state of a queue in an egress buffer associated with the packet, to be processed by the processor associated with the selected ingress buffer to provide a network service

BACKGROUND

The demand for network devices to provide network services is growing.Such network services may include data compression and caching(including molecular sequence reduction and network sequence caching),acceleration (including packet flow acceleration and application flowacceleration), application control (including quality of service (QoS)and policy-based multipath), and deterministic finite automaton (DFA)pattern matching.

SUMMARY

According to one aspect, a method may include receiving a packet in anetwork device; selecting one of a group of ingress buffers, where eachingress buffer is associated with a different one of a group ofprocessors; distributing the packet to the selected ingress buffer; andscheduling the packet, based on a congestion state of a queue in anegress buffer associated with the packet, to be processed by theprocessor associated with the selected ingress buffer to provide anetwork service.

According to another aspect, a network device may include a receiver toreceive a packet; a group of memories, each memories storing a differentingress buffer and at least one of memories storing an egress bufferincluding a queue associated with the packet; and a group of processors,each associated with a different ingress buffer, where one of theprocessors selects one of a plurality of ingress buffers and distributesthe packet to the selected ingress buffer; where the processorassociated with the selected ingress buffer schedules the packet, basedon a congestion state of the queue associated with the packet, to beprocessed by the processor associated with the selected ingress bufferto provide a network service.

According to another aspect, an apparatus may include means forreceiving a packet in a network device; means for selecting one of aplurality of ingress buffers, where each ingress buffer is associatedwith a different one of a plurality of processors; means fordistributing the packet to the selected ingress buffer; and means forscheduling the packet, based on a congestion state of a queue in anegress buffer associated with the packet, to be processed by theprocessor associated with the selected ingress buffer to provide anetwork service.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate one or more embodiments describedherein and, together with the description, explain these embodiments. Inthe drawings,

FIG. 1 is a block diagram of an exemplary environment in whichembodiments described herein may be implemented;

FIG. 2 is a block diagram of exemplary components of the network deviceof FIG. 1;

FIG. 3 is a block diagram of exemplary functional elements of thenetwork device of FIG. 1;

FIG. 4 is a block diagram of exemplary functional elements of thenetwork device of FIG. 1 including a group of modules;

FIG. 5 is a flowchart of an exemplary process for providing a networkservice in the network device of FIG. 1;

FIG. 6 is a flowchart of an exemplary process for scheduling traffic inthe network device of FIG. 1; and

FIG. 7 is a flowchart of an exemplary process for signaling congestionin the network device of FIG. 1.

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawings.The same reference numbers in different drawings may identify the sameor similar elements. Also, the following detailed description does notlimit the invention. Instead, the scope of the invention is defined bythe appended claims and equivalents.

As the demand for network services increases, network devices thatprovide these services may be limited by processing power, among otherthings. In embodiments disclosed herein, a network device may receive apacket and may select one of a group of ingress buffers for distributingthe packet. Each ingress buffer may be associated with a different oneof a group of processors. The network device may then distribute thepacket to the selected ingress buffer and schedule the packet forprocessing in the associated processor based on the congestion state ofa queue in an egress buffer for the network device, for example. Assuch, the network device may properly allocate processor resources andegress buffer resources.

Exemplary Environment

FIG. 1 is a block diagram of an exemplary environment 100 in whichembodiments described herein may be implemented. Environment 100 mayinclude a network 102, a router 104, a switch 106, a local area network(LAN) 108, a node 109, network devices 110-1 and 110-2, a router 112, aswitch 114, a LAN 116, and a node 118. In practice, there may be more,different, or fewer devices or a different arrangement of devices thanwhat is shown in FIG. 1. For example, environment 100 may includethousands or even millions of nodes and/or network devices. Further,while FIG. 1 shows devices in environment 100, one or more of thesedevices may be remotely located from each other, e.g., the devices maybe geographically diverse. Although arrows in FIG. 1 may indicatecommunication directly between devices, communication may be indirectthrough one or more networks.

Communication among network 102, router 104, switch 106, local areanetwork (LAN) 108, node 109, network devices 110-1 and 110-2, router112, switch 114, LAN 116, and node 118 may be accomplished via wiredand/or wireless communication connections. Node 109 (coupled to LAN 108)may communicate with node 118 (coupled to LAN 116) and vice versa.Communication between nodes 109 and 118 may take place through LAN 108,switch 106, network device 110-1, router 104, network 102, router 112,network device 110-2, switch 114, and LAN 116. For example, node 109 mayestablish a session with node 118. Sessions may include telephone calls,multimedia distribution, or multimedia conferences. A session mayinclude a lasting connection between two nodes and may include one ormore flows of packets between nodes. For example, a telephone callbetween nodes 109 and 118 may include a real-time flow of packets fromnode 109 to node 118 and a real-time flow of packets from node 118 tonode 109.

Network 102 may include a wide-area network (WAN), e.g., the Internet, alocal-area network (either wired or wireless), a telephone network,e.g., the Public Switched Telephone Network (PSTN), an intranet, aprivate corporate network, or a combination of networks.

Routers 104 and 112 and switches 106 and 114 may inspect packets,determine the proper path for the packets, and forward the packets toother devices next in line along the determined paths. LANs 109 and 118may include a group of computers or other devices, including nodes 109and 118, respectively, that may communicate among each other.

Nodes 109 and 118 may include computers, telephones, personal digitalassistants, or any other communication devices that may transmit orreceive data. Nodes 109 and 118 may include, for example, computers thatexchange data

Network devices 110-1 and 110-2 may provide network services to nodes inenvironment 100. Such services may include compression and caching(including molecular sequence reduction and network sequence caching),acceleration (including packet flow acceleration and application flowacceleration), application control (including quality of service (QoS)and policy-based multipath), and DFA pattern matching. Network devices110-1 and 110-2 may receive packets, perform network services, and thenforward packets. Network devices 110-1 and 110-2 may also include andperform the functions of a router, a switch, a packet forwarding engine,a firewall, or any other network device capable of receiving andforwarding packets. Network devices 110-1 and 110-2 may include, forexample, Juniper's WX system™ and Juniper's IDP™ system.

Network Device

FIG. 2 is a block diagram of exemplary components of network devices110-1 and 110-2 (each network device may be referred to as “networkdevice 110”). As shown in FIG. 2, network device 110 may include amodule 200. In one embodiment, network device 110 may include a “rack”that may include one or more modules, e.g., a cluster of modules. Module200 may include a bus 210, processing logic 220, a communicationinterface 230, and a memory 240. Module 200 may include other components(not shown) that aid in receiving, transmitting, and/or processing data.Moreover, other configurations of components in module 200 are possible.In addition, network device 110 may include other components (not shown)or configuration of components.

Bus 210 may include a path that permits communication among thecomponents of module 200. Processing logic 220 may include any type ofprocessor or microprocessor that interprets and executes instructions.In other embodiments, processing logic 220 may include an applicationspecific integrated circuit (ASIC), field programmable gate array(FPGA), or the like.

Communication interface 230 may include any transceiver-like mechanism(e.g., a receiver/transmitter combination) that enables module 200 tocommunicate with other devices and/or systems. Communication interface230 may allow for wired or wireless communications. In oneimplementation, communication interface 230 may allow for module 200 tobe controlled and/or administered remotely by an operator or anadministrator.

Memory 240 may include a random access memory (RAM) or another type ofdynamic storage device that may store information and instructions forexecution by processing logic 220; a read only memory (ROM) device oranother type of static storage device that may store static informationand instructions for use by processing logic 220; and/or some other typeof magnetic or optical recording medium and its corresponding drive forstoring information and/or instructions. Memory 240 may store networkservice applications. Network service applications may includeinstructions for causing module 200 to implement network services, suchas compression and caching (including molecular sequence reduction andnetwork sequence caching), acceleration (including packet flowacceleration and application flow acceleration), application control(including quality of service (QoS) and policy-based multipath), and DFApattern matching.

Module 200 may perform certain operations, as described in detail below.Module 200 may perform these operations in response to processing logic220 executing software instructions contained in a computer-readablemedium, such as memory 240. A computer-readable medium may be defined asa physical or logical memory device and/or carrier wave. The softwareinstructions may be read into memory 240 from another computer-readablemedium or from another device via communication interface 230. Thesoftware instructions contained in memory 240 may cause processing logic220 to perform processes that are described below.

FIG. 3 is a block diagram of exemplary functional elements of networkdevice 110. As with FIG. 2, network device 110 in FIG. 3 may include amodule 200. In one embodiment, network device 110 may include a rackthat may include one or more modules. Module 200 may include a trafficclassifier 302, an ingress buffer 304, a packet scheduler 306, aprocessor 308, an egress buffer 310, and a traffic shaper 312. Inpractice, there may be more, different, or fewer components or adifferent arrangement of components than what is shown in FIG. 3.Further, although FIG. 3 shows components within module 200, componentsmay be remotely located from each other, e.g., the components may begeographically diverse. Although arrows in FIG. 3 may indicatecommunication directly between components, communication may be indirectthrough one or more networks.

Traffic classifier 302 may inspect packets as they arrive and mayclassify the packets. In one embodiment, traffic classifier 302 mayinspect the source address, destination address, source port,destination port, and/or protocol of a packet to determine whether thepacket belongs to a class, flow, and/or session. Traffic classifier 302may also inspect deeper into a packet (e.g., layer 7 inspection) todetermine a class, flow, and/or a session. For example, node 109 mayestablish a session with node 118 and the session may include tworeal-time flows of packets (e.g., voice communications) in eachdirection between nodes 109 and 118. If a packet is determined not to bea part of a previously-identified flow, traffic classifier 302 maydetermine that the packet is part of a newly-identified flow. If it isdetermined that a packet is part of a previously-identified flow,traffic classifier 302 may determine that the packet is part of thispreviously-identified flow.

Ingress buffer 304 may store packets until packet scheduler 306schedules them for release to processor 308. Ingress buffer 304 mayinclude many individual queues, and an individual queue may beidentified by a queue ID. Queuing in ingress buffer 304 may behierarchical with multiple levels. At one level in one embodiment, theremay be a queue for each logical interface. At another level in oneembodiment, each class of traffic may have queue. At yet another levelin one embodiment, each flow of traffic may have a queue. For example,ingress buffer 304 may include a queue for each class of trafficprovided by traffic classifier 302. In one embodiment, ingress buffer304 may also perform policing, rate limiting, and marking.

Packet scheduler 306 may determine when packets are released fromingress buffer 304 to processor 308. In one embodiment, packet scheduler306 may throttle, e.g., delay, the release of a packet if the egressqueue that buffers the post-processed packet is congested. For example,packet scheduler 306 may throttle the release of packets belonging to aclass of traffic, for example, if a queue (or queues) storing packetscorresponding to that class is congested in egress buffer 310. Packetscheduler 306 may receive congestion information from egress buffer 310through a communication line 314 (shown as a dashed line in FIG. 3).Packet scheduler 306 may use different policies to schedule packets,such as a priority-based policy or a weighted-sharing based policy. Inone embodiment, the policy to schedule packets may be selected by auser, e.g., a customer or service provider. Packet scheduler 306 mayassist in properly allocating the resources of processor 308 and egressbuffer 310.

Each traffic class may be treated differently to differentiate services.For example, each traffic class may be subject to a different ratelimit, shaped separately and/or prioritized relative to other trafficclasses. Upon classifying a traffic flow using a particular protocol, adifferent policy may be applied to different classes of traffic toguarantee a quality of service (QoS). For example, VoIP traffic or mediastreaming traffic may require a minimum QoS and email traffic mayrequire best-effort delivery. Traffic classifier may identify sessions,flows, etc., by using a class ID. Upon classification, trafficclassifier 302 may pass packets to ingress buffer 304, along with theclass ID of the packets.

Processor 308 may provide network service applications to nodes inenvironment 100. Services may include compression and caching (includingmolecular sequence reduction and network sequence caching), acceleration(including packet flow acceleration and application flow acceleration),application control (including quality of service (QoS) and policy-basedmultipath), and DFA pattern matching. Other types of network servicesare possible. For example, other services may include servicesassociated with switches, routers, bridges, and/or hubs. Processor 308may pass post-processed packets to egress buffer 310. A processed packetmay include compressed data and, therefore, may include information thatwas in multiple pre-processed packets. As a result, in one embodiment,processor 308 may pass post-processed packets to egress buffer 310without delay to minimize latency.

Egress buffer 310 may store packets until traffic shaper 312 releasesthem for transmission out of network device 110 toward theirdestination. Egress buffer 310 may include a group of queues. The queuesin egress buffer 310 may be structured similarly to the queues iningress buffer 304. Queuing in egress buffer 302 may be hierarchicalwith multiple levels. At one level in one embodiment, there may be aqueue for each logical interface. At another level in one embodiment,each class of traffic may have queue. At yet another level in oneembodiment, each flow of traffic may have a queue. In one embodiment,there is a queue in egress buffer 310 corresponding to each queue iningress buffer 304. Traffic shaper 312 and egress buffer 310 may performtraffic shaping, e.g., they may attempt to control traffic in order tooptimize or guarantee performance, low latency, and/or bandwidth bydelaying and/or scheduling packets.

FIG. 4 is a block diagram of exemplary functional elements of networkdevice 110. As shown in FIG. 4, network device 110 may include a groupof modules, including modules 200-1 through module 200-3 (individually“modules 200-x”). In one embodiment, module 200-1 may be considered a“master” module and modules 200-2 and 200-3 may be considered “slave”modules.

Modules 200-1 through 200-3 may be configured similarly to module 200 inFIG. 2 and FIG. 3. Module 200-1 may include a traffic classifier 302-1,an ingress buffer 304-1, a packet scheduler 306-1, a processor 308-1, anegress buffer 310-1, and a traffic shaper 312-1. Module 200-2 mayinclude an ingress buffer 304-2, a packet scheduler 306-2, and aprocessor 308-2. Module 200-3 may include an ingress buffer 304-3, apacket scheduler 306-3, and a processor 308-3. In some implementations,modules 200-2 and 200-3 may also include traffic classifiers and/oregress buffers, but these functional elements are not shown in FIG. 4for simplicity.

In practice, there may be more, different, or fewer modules or adifferent arrangement of modules than what is shown in FIG. 4. Further,modules 200-1 through 200-3 may be remotely located from each other,e.g., the modules 200-1 through 200-3 may be geographically diverse.Although arrows in FIG. 4 may indicate communication directly betweenmodules, communication may be indirect through one or more networks.

A packet may enter module 200-1 and may enter traffic classifier 302-1.Traffic classifier 302 may forward the packet to ingress buffers 304-1,304-2, or 304-3. As such, traffic classifier 302-1 may perform thefunctions of a load balancer, distributing received packets amongmodules 200-1, 200-2, and 200-3. Traffic classifier 302-1 may balancethe load based on determined flows, classes, or sessions, for example.

Ingress buffers 304-1, 304-2, and 304-3 may have hierarchical ingressqueues similar to the ingress queues described with respect to ingressbuffer 304 above. Packet scheduler 306-1 may release packets toprocessor 308-1 based on the congestion status of the correspondingqueue in egress buffer 310-1. For example, if egress buffer 310-1 iscongested for a particular queue, then the corresponding queue iningress buffer 304-1 may, in one embodiment, not release a packet toprocessor 308-1 Likewise, packet scheduler 306-2 may release packets toprocessor 308-2 based on the congestion status of the correspondingqueue in egress buffer 310-1. Further, packet scheduler 306-3 mayrelease packets to processor 308-3 based on the congestion status of thecorresponding queue in egress buffer 310-1. Packet schedulers 306-1,306-2, and 306-3 may receive congestion state information from egressbuffer via a communication channel (shown with dashed lines in FIG. 4).

Processors 308-1 through 308-3 may perform network services describedabove with respect to processor 308 and FIG. 3.

Egress buffer 310-1 may store packets until traffic shaper 312-1 mayrelease them for transmission out of network device 110 toward theirdestination. Traffic shaper 312-1 and egress buffer 310-1 may performtraffic shaping, e.g., they may attempt to control traffic in order tooptimize or guarantee performance, low latency, and/or bandwidth bydelaying packets. In this embodiment, traffic is shaped by egress buffer310-1 and traffic shaper 312-1 only.

In another embodiment, egress buffer 310-2 (not shown) and egress buffer310-3 (not shown) may be used for traffic shaping. In this embodimentmodules 200-2 and 200-3 may maintain their own egress buffers andtraffic shaping may either not be uniformly performed by the modules200-1 through 200-3 or modules 200-1 through 200-3 may communicate witheach other for uniform traffic shaping.

Exemplary Processes

FIG. 5 is a flowchart of an exemplary process 500 for providing anetwork service. Process 500 may run in network device 110. Process 500is described below with respect to network device 110 in FIG. 4including modules 200-1 through 200-3.

Process 500 may begin when network device 110 receives a packet (block502). The packet may be classified and forwarded to the appropriatemodule (block 504). Packets may be classified and forwarded by trafficclassifier 302-1, for example. In one embodiment, packets in the sameclass, flow, and/or session may be forwarded to the same module and maybe placed in the same queue. In another embodiment, packets may beforwarded to modules 200-1 through 200-3 in a round-robin fashion on aper-flow basis.

The congestion state may be checked and the packet may be scheduled(block 506). For example, if the packet is distributed to module 200-1,then packet scheduler 306-1 may check the congestion state and schedulethe packet. If the packet is distributed to module 200-2, then packetscheduler 306-2 may check the congestion state and schedule the packet.If the packet is distributed to module 200-3, then packet scheduler306-3 may check the congestion state and schedule the packet. The packetmay be scheduled based on congestion information received from egressbuffer 310. For example, if the egress queue corresponding to theingress queue storing the packet is congested, then the ingress queuemay not schedule the packet to be sent to the processor. Scheduling apacket (block 506) may allow for a proper allocation of systemresources, such as the resources of processor 308 and egress buffer 310.Scheduling a packet (block 506) may also allow for different schedulingpolicies to apply to different flows, classes and/or sessions. Packetscheduler 306-1, 306-2, or 306-3 may use policies to schedule packets,such as a priority-based policy or a weighted-sharing based policy. Forexample, a packet in a flow of packets in a telephone conversation(requiring a minimum QoS) may be scheduled before a packet associatedwith an e-mail (requiring a best effort).

When the packet is sent to the processor (e.g., processor 306-1, 306-2,or 306-3), the packet may be processed (block 508). The post-processedpacket may be sent to egress buffer 310-1 (block 510) for trafficshaping. Traffic shaper 312-1 may schedule and sending the packet out ofnetwork device 110 (block 512).

FIG. 6 is a flowchart of an exemplary process 600 for schedulingtraffic. Process 600 is described below with respect to network device110 as shown in FIG. 4 including modules 200-1, 200-2, and 200-3.Process 600 may run in network device 110 and may schedule packets beingreleased from an ingress queue, such as an ingress queue in ingressbuffers 304-1, 304-2, and 304-3. Portions of or all of process 600 mayrun in modules 200-1, 200-2, and/or 200-3. In one embodiment, process600 may run in packet scheduler 306-1. Process 600 is described withrespect to ingress buffer 304-2, packet scheduler 306-2, and processor308-2 in module 200-2. Process 600 may equally apply to the componentsand/or functional elements of modules 200-1 and 200-3.

The congestion state of a queue in egress buffer 310-1 may be receivedand the time the congestion state was received may be marked (block602). If the state is NOT CONGESTED (block 604: NO), then packetscheduler 306-2 may schedule packets (block 612) in the correspondingingress queue in ingress buffer 304-2 for release to processor 308-2. Ifthe state is CONGESTED (block 604: YES), then packet scheduler 306-2 maynot schedule packets in the corresponding ingress queue in ingressbuffer 304-2 for release to processor 308-2. Releasing a packet toprocessor 308-2 when its egress queue is CONGESTED may be a waste of theresources of processor 308-2 because processor 308-2 may release theprocessed packet to egress buffer 310-1 where the processed packet mustwait.

If the state is CONGESTED (block 604: YES), but the current time minusthe marked time when the congested state was received (ΔT) is greaterthan a threshold (T1), then the state stored for the egress queue inegress buffer 310-1 may be reset to NOT CONGESTED (block 610). If thecurrent time minus the time the congested state was received (ΔT) is notgreater than a threshold T1, then process 600 may wait until sufficienttime has passed such that the threshold T1 has been reached (e.g.,process 600 may loop back to block 608) before the state may be reset toNOT CONGESTED (block 610). However, if a congestion state is receivedfrom the egress queue in egress buffer 310-1, process 600 may beinterrupted and may start again at block 602. Resetting the state to NOTCONGESTED (block 610) may allow for the scheduling of packets (block612) in the event that a NOT CONGESTED state message is lost betweenegress buffer 310-1 and packet scheduler 306-2. Threshold T1 may be onthe order of a millisecond, a centisecond, a second, etc.

FIG. 7 is a flowchart of an exemplary process 700 for signalingcongestion in network device 110. Process 700 is described with respectto network device 110 shown in FIG. 4 including modules 200-1, 200-2,and 200-3. In particular, process 700 is described with respect toingress buffer 304-2, packet scheduler 306-2, and processor 308-2 inmodule 200-2. Process 700 may equally apply to the components and thefunctional elements of modules 200-1 and 200-3. In one embodiment,process 700 may run in network device 110 as part of egress buffer310-1.

Process 700 may begin when egress buffer 310-1 sends a packet from aqueue or receives a packet for a queue (block 702). The queue lengthcorresponding to the queue having sent or received the packet may bedetermined (block 702). If the queue length is greater than a thresholdHIGH WATER MARK (block 704: YES), then the state for that queue maybecome CONGESTED (block 706) and process 700 may proceed to block 712.If the queue length is not greater than the threshold HIGH WATER MARK(block 704: NO), then process 700 may proceed to block 708 where it maybe determined whether the queue length is less than the threshold LOWWATER MARK (block 708). If so (block 708: YES), then the state for thequeue may become NOT CONGESTED (block 710) and procedure 700 may proceedto block 712. In one embodiment, the HIGH WATER MARK may be greater thanthe LOW WATER MARK. In one embodiment, the HIGH WATER MARK and the LOWWATER MARK can each be different for different queues. Having the twothresholds HIGH WATER MARK and LOW WATER MARK may allow for a hystorisisand may reduce the amount of signaling of the congestion state toingress buffers 304-1, 304-2, and 304-3.

If the congestion state has changed (block 712: NO) since the previoustime process 700 was performed (e.g., comparing the current state to thevalue stored in variable PREVIOUS STATE), then the congestion state maybe sent to the modules (block 716), e.g., packet schedulers 306-1,306-2, and 306-3, and the time of sending the congestion state may bemarked (block 718).

If the congestion state has not changed (block 712: YES), then thecurrent state is sent to all modules (block 716) when the current timeminus the previously marked time (ΔT) is greater than a threshold T2(block 714: YES) and process 700 may move to block 718. If thecongestion state has not changed (block 712: YES) and the current timeminus the previously marked time (ΔT) is not greater than the thresholdT2 (block 714: NO), then process 700 may move to block 718. At block 718the variable PREVIOUS STATE is set to value of the current state for thenext time process 700 may be performed.

In one embodiment, T2 is less than T1 so that ingress buffer 304-2 maybe updated with the congestion state before the current state “timesout” in ingress buffer 304-2. In one embodiment, the queue length ischecked only when a packet is sent or received because any congestedstate will expire in egress buffers 310-1 through 310-3. In anotherembodiment, the queue length is checked on a periodic basis.

Conclusion

One or ore embodiments described herein may efficiently use limitedegress buffer resources in a data traffic processing system. One or moreembodiments described herein may efficiently use limited processorresources in a traffic processing system. Embodiments described hereinmay provide for centralized egress buffers and centralized trafficshaping. One or more embodiments described herein allow for robust andfault tolerant protocol to signal congestion in a network device.

The descriptions of exemplary components above include a discussion ofsoftware instructions contained in computer-readable media.Alternatively, in each of these implementations, hardwired circuitry maybe used in place of or in combination with software instructions toimplement processes described herein. Thus, implementations describedherein are not limited to any specific combination of hardware circuitryand software.

It will also be apparent that aspects, as described above, may beimplemented in many different forms of software, firmware, and hardwarein the implementations illustrated in the figures. The actual softwarecode or specialized control hardware used to implement these aspects isnot limiting of the present invention. Thus, the operation and behaviorof the aspects were described without reference to the specific softwarecode—it being understood that software or control hardware could bedesigned to implement the aspects based on the description herein.

Further, although the processes described above, including process 500,600, and 700, may indicate a certain order of blocks, the blocks inthese figures may be configured in any order.

In addition, implementations described herein may use theinternet-protocol (IP), asynchronous transfer mode (ATM) protocol, orany other type of network protocol. As such, implementations describedherein may use IP addresses, ATM addresses, or any other type of networkaddresses. Implementations may be described in terms of packets,implementations could use any form of data (packet or non-packet).

No element, act, or instruction used in the present application shouldbe construed as critical or essential to the invention unless explicitlydescribed as such. Also, as used herein, the article “a” is intended toinclude one or more items. Where only one item is intended, the term“one” or similar language is used. Further, the phrase “based on” isintended to mean “based, at least in part, on” unless explicitly statedotherwise.

1-20. (canceled)
 21. A method comprising: receiving a packet in anetwork device; selecting one of a plurality of ingress buffers;distributing the packet to the selected ingress buffer; receiving, froman egress buffer associated with the packet, a congestion state of aqueue in the egress buffer; determining whether to schedule the packetto be processed by a processor associated with the selected ingressbuffer based on whether the received congestion state indicates that thequeue in the egress buffer is congested and the congestion state wasreceived not more than a threshold amount of time earlier; andprocessing, by the processor associated with the selected ingressbuffer, the packet when the packet is scheduled to be processed by theprocessor.
 22. The method of claim 21, where determining whether toschedule the packet to be processed further includes not permitting theprocessing of the packet when the received congestion state indicatesthat the queue in the egress buffer is congested and the congestionstate was received not more than a threshold amount of time earlier. 23.The method of claim 21, where determining whether to schedule the packetfurther includes: determining to schedule the packet to be processedwhen the congestion state was received more than a threshold timeearlier and the received congestion state indicates that the queue inthe egress buffer is congested.
 24. The method of claim 21, whereselecting the one of the plurality of ingress buffers includes:classifying the packet based on one or more of a source address, adestination address, a source port, a destination port, a protocol, oran application associated with the packet; and selecting the one of theplurality of ingress buffers based on the classifying.
 25. The method ofclaim 24, where classifying the packet includes determining whether thepacket belongs to a class, a flow, or a session; and where distributingthe packet includes distributing the packet based on the determinationof whether the packet belongs to the class, the flow, or the session.26. The method of claim 21, further comprising: determining a length ofthe queue in the egress buffer; indicating the congestion state ascongested when the length is greater than a first threshold; andindicating the congestion state as not congested when the length is lessthan a second threshold.
 27. The method of claim 26, further comprisingreceiving the congestion state of the queue in the egress buffer whenthe state changes or periodically when the state does not change.
 28. Anetwork device comprising: one or more processors to: receive a packet;select one of a plurality of ingress buffers; distribute the packet tothe selected ingress buffer; receive, from an egress buffer associatedwith the packet, a congestion state of a queue in the egress buffer;determine whether to schedule the packet to be processed by a processorassociated with the selected ingress buffer based on whether thereceived congestion state indicates that the queue in the egress bufferis congested and the congestion state was received not more than athreshold amount of time earlier; and process the packet when the packetis scheduled to be processed.
 29. The network device of claim 28, where,when determining whether to schedule the packet to be processed, the oneor more processors are further to not schedule the packet to beprocessed when the received congestion state indicates that the queue inthe egress buffer is congested and the congestion state was received notmore than a threshold amount of time earlier.
 30. The network device ofclaim 28, where, when determining whether to schedule the packet, theone or more processors are further to: determine to schedule the packetto be processed when the congestion state was received more than athreshold time earlier and the received congestion state indicates thatthe queue in the egress buffer is congested.
 31. The network device ofclaim 28, where, when selecting the one of the plurality of ingressbuffers, the one or more processors are further to: classify the packetbased on one or more of a source address, a destination address, asource port, a destination port, a protocol, or an applicationassociated with the packet; and select the one of the plurality ofingress buffers based on the classifying.
 32. The network device ofclaim 31, where, when classifying the packet, the one or more processorsare further to determine whether the packet belongs to a class, a flow,or a session; and where, when distributing the packet, the one or moreprocessors are further to distribute the packet based on thedetermination whether the packet belongs to the class, the flow, or thesession.
 33. The network device of claim 28, where the one or moreprocessors are further to: determine a length of the queue in the egressbuffer; indicate the congestion state as congested when the length isgreater than a first threshold; and indicate the congestion state as notcongested when the length is less than a second threshold.
 34. Thenetwork device of claim 33, where the one or more processors are furtherto receive the congestion state of the queue in the egress buffer whenthe state changes or periodically when the state does not change. 35.The network device of claim 33, where, when determining whether toschedule the packet to be processed, the one or more processors arefurther to: hold the packet in the selected one of the plurality ofingress buffers without scheduling the packet for processing when thecongestion state was received not more than the threshold time earlierand the received congestion state indicates that the queue in the egressbuffer is congested.
 36. The network device of claim 28, where the oneor more processors are further to: output the processed packet to adestination of the processed packet, where the destination is anothernetwork device.
 37. A network device comprising: a receiver to receive apacket; a plurality of memories, each of the plurality of memoriesstoring a different ingress buffer and at least one of the plurality ofmemories storing an egress buffer, associated with a plurality of thedifferent ingress buffers, the egress buffer including a queueassociated with the packet; and a plurality of processors, eachprocessor associated with a different ingress buffer, a first processor,of the plurality of processors, to: select one of the plurality ofdifferent ingress buffers, and distribute the packet to the selectedingress buffer, a processor associated with the selected ingress bufferto: receive, from an egress buffer associated with the packet, acongestion state of a queue in the egress buffer; determine whether toschedule the packet to be processed based on whether the receivedcongestion state indicates that the queue in the egress buffer iscongested and the congestion state was received not more than athreshold amount of time earlier; and process the packet when the packetis scheduled to be processed.
 38. The network device of claim 37, where,when determining whether to schedule the packet to be processed, theprocessor associated with the selected ingress buffer is further to notschedule the packet to be processed when the received congestion stateindicates that the queue in the egress buffer is congested and thecongestion state was received not more than a threshold amount of timeearlier.
 39. The network device of claim 37, where, when determiningwhether to schedule the packet to be processed, the processor associatedwith the selected ingress buffer is further to: determine to schedulethe packet to be processed when the congestion state was received morethan a threshold time earlier and the received congestion stateindicates that the queue in the egress buffer is congested.
 40. Thenetwork device of claim 37, where, when selecting the one of theplurality of ingress buffers, the first processor is further to:classify the packet based on one or more of a source address, adestination address, a source port, a destination port, a protocol, oran application associated with the packet; and select the one of theplurality of ingress buffers based on the classifying.